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1. Introduction 

Before discussing the main results of the paper, we briefly describe the few basic 
notions of Nielsen fixed point theory which will be used. We assume X to be a 
connected, compact polyhedron and f : X —>■ X to he a continuous map. Let 
p : X ^ X he the universal cover of X and f : X —* X a lifting of /, i.e. pof = fop- 
Two liftings / and /' are called conjugate if there is a 7 G F = 7ri(X) such that 
/' = 7 o / o 7~^. The subset p{Fix{f)) C Fix{f) is called the fixed point class of f 
determined by the hfting class [/].Two fixed points Xq and Xi of / belong to the same 
fixed point class iff there is a path c from xq to Xi such that c = / o c (homotopy 
relative endpoints). This fact can be considered as an equivalent definition of a 
non-empty fixed point class. Every map / has only finitely many non-empty fixed 
point classes, each a compact subset of X. A fixed point class is called essential 
if its index is nonzero. The number of essential fixed point classes is called the 
Nielsen number of /, denoted by A^(/).The Nielsen number is always finite. A^(/) 
is a homotopy type invariant. In the category of compact, connected polyhedra, the 
Nielsen number of a map is, apart from certain exceptional cases, equal to the least 
number of fixed points of maps with the same homotopy type as /. 

Recently a connection between symplectic Floer homology and Nielsen fixed point 
theory was discovered [TU [2D] • The author came to the idea that Nielsen fixed point 
theory is connected with symplectic Floer homology theory of symplectomorphisms 
of surfaces at the Autumn 2000, after conversations with Joel Robbin and Dan 
Burghelea. This connection is based on the following simple fact: Gromov pseu- 
doholomorphic curve in symplectic fixed point theory is nothing else as a Nielsen 
holomorphic disc. In the dimension two a diffeomorphism is symplectic if it preserves 
area. As a consequence, the symplectic geometry of surfaces lacks many of the in- 
teresting phenomena which are encountered in higher dimensions. For example, two 
symplectic automorphisms of a closed surface are symplectically isotopic iff they 
are homotopic, by a theorem of Moser [33|- On other hand symplectic fixed point 
theory is very nontrivial in dimension 2. A selebrated landmark in this subject is 
Poincare's last geometric theorem, proved by Birkhoff pQ in the 1925, which asserts 
that an area-preserving twist map of the annulus must have at least two distinct 
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fixed points. It is known that symplectic Floer homology for surface symplectomor- 
phism is the d = 1 part of periodic Floer homology of this symplectomorphism [23] , 
a Floer homology theory whose chain complex is generated by certain multisets of 
periodic orbits and whose differentials count certain embedded pseudoholomorphic 
curves. This theory is conjectured [23] to be isomorphic to the Seiberg - Witten 
Floer homology |33j of the mapping torus of symplectomorphism in certain spin'^- 
structures and also is conjectured to agree with Heegaard Floer homology HF~^ [38] 
of the mapping torus. It is known also that symplectic Floer homology of symplec- 
tomorphism of surface is a simple model for the instanton Floer homology of the 
mapping torus of this symplectomorphism [B] . 

In the chapter [2] we discuss the connection between symplectic Floer homology 
theory and Nielsen fixed point theory and the calculations of Seidel's symplectic 
Floer homology for periodic [20l [T3] and algebraically finite mapping classes [20] . In 
the chapter O we describe symplectic zeta functions and an asymptotic invariant of 
monotone symplectomorphism. In the chapter |3] we discuss a generalisation of the 
Poincare - Birkhoff theorem and Arnold conjecture and the calculations of Seidel's 
symplectic Floer homology for pseudo-Anosov and reducible mapping classes [3| 
The results of this paper were announced on the Symplectic Geometry Seminars at 
Stanford and Princeton in April and October, 2007. 

The author is grateful to A. Cotton-Clay, Ya. Eliashberg, Wu-Chung Hsiang, 
E. lonel, J. Kedra, K. Ono, Y. Rudyak, G. Tian, V. Turaev and M. Usher for 
stimulating discussions and comments. 

Parts of this article were written while the author was visiting the Max-Planck- 
Institute fiir Mathematik, Bonn in 2004-2005. The author would like to thank the 
Max-Planck-Institute fiir Mathematik, Bonn for kind hospitality and support. 

2. Nielsen fixed point theory and symplectic Floer homology 
2.1. Symplectic Floer homology. 

2.1.1. Monotonicity. In this section we discuss the notion of monotonicity as defined 
in [311 [20] . Monotonicity plays important role for Floer homology in two dimensions. 
Throughout this article, M denotes a closed connected and oriented 2-manifold of 
genus > 2. Pick an everywhere positive two-form u on M. 
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Let G Synip(M, u), the group of symplectic automorphisms of the two-dimensional 
symplectic manifold {M,uj). The mapping torus of 0, T^ = M x M/{t + l,x) ~ 
(t, 0(x)), is a 3-manifold fibered over S^ = M/Z. There are two natural second co- 
homology classes on T^, denoted by [uj^] and c^. The first one is represented by the 
closed two-form u^ which is induced from the puUback of cj to M x M. The second 
is the Euler class of the vector bundle V^ = M x TM/{t + l,^x) ~ {t, d(j)r^C,x), which 
is of rank 2 and inherits an orientation from TM. 

(j) e Symp(M, Lj) is called monotone, if [u^] = (area^(M)/x(M))-c<^ in H'^(T^; M); 
throughout this article Symp'"(M, cu) denotes the set of monotone symplectomor- 
phisms. 

Now H^IT^;^) fits into the following short exact sequence [HI [20] 



(2.1) — > . ^'^f:^l, -^ H^{T,; R) ^ H\M, M), ^ 0. 

image(id— 0*) 

where the map r* is restriction to the fiber. The map d is defined as follows. Let 
p : J ^ M be a smooth function which vanishes near and 1 and satisfies J„ p dt = 1. 
If 6' is a closed 1-form on M, then p ■ 6 A dt defines a closed 2-form on T^; indeed 
d[9] = [p ■ 9 A dt]. The map r : M ^^ T^ assigns to each x E M the equivalence 
class of (1/2, x). Note, that r*uj^ = u and r*c^ is the Euler class of TM. Hence, 
by ( 12. ip . there exists a unique class m(0) G //""^(M; M)/ image (id —0*) satisfying 
dm{(j)) = [uj^] — (area^(M)/x(M)) ■ c^, where x(^) denotes the Euler characteristic 
of M. Therefore, is monotone if and only if ?7i(0) = 0. 

Because c^ controls the index, or expected dimension, of moduli spaces of holo- 
morphic curves under change of homology class and u^ controls their energy under 
change of homology class, the monotonicity condition ensures that the energy is con- 
stant on the index one components of the moduli space, which implies compactness 
and, as a corollary, finite count in a differential of the Floer complex. 

We recall the fundamental properties of Symp'"(M, on) from [HHSU]- Let Diff"'"(M) 
denotes the group of orientation preserving diffeomorphisms of M. 

(Identity) idA/ G Symp'"(M,u;). 

(Naturality) If G Symp'"(M,u;),7/; G Diff+(M), then ^"V^ e Symp'"(M,^*cj). 

(Isotopy) Let {tpt)t(^i be an isotopy in Symp(M, a;), i.e. a smooth path with tpo = id. 
Then m(0o'0i) = m(0) + [F\ux{4't)t€i] in H^{M; M)/ image(id —0*); see [H], Lemma 
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6] . For the definition of tlie flux homomorphism see [35] . 

(Inclusion) The inclusion Symp™'(M, cu) "^^ Diff'''(M) is a homotopy equivalence. 

(Floer homology) To every G Synip'^(M, cj) symplectic Floer homology theory 
assigns a Z2-graded vector space HF^{(f)) over Z2, with an additional multiplicative 
structure, called the quantum cap product, H*{M; Z2) ® HF^{(j)) — > HF^{(j)). For 
= idjv/ the symplectic Floer homology HF^,{i(\m) are canonically isomorphic to 
ordinary homology H^{M]'L2) and quantum cap product agrees with the ordinary 
cap product. Each ip G Diff'^(M) induces an isomorphism HF^{(j)) = H F^{ip~^ (pip) 
of i7*(M;Z2)-modules. 

(Invariance) If 0,0' G Symp'"(M, cj) are isotopic, then HF^,{(I)) and HF^{(j)') are 
naturally isomorphic as iif*(M; Z2)-modules. This is proven in [HI Page 7]. Note 
that every Hamiltonian perturbation of (see [S]) is also in Symp'"(M, cu). 

Now let g he a. mapping class of M, i.e. an isotopy class of Diff''"(M). Pick an area 
form u and a representative G Symp™'(M, cj) of g. Then HF^{(j)) is an invariant 
of g, which is denoted by HF^{g). Note that HF^:{g) is independent of the choice of 
an area form u by Moser's isotopy theorem [37] and naturality of Floer homology. 

2.1.2. Floer homology. Let G Symp(M,ci;). There are two ways of constructing 
Floer homology detecting its fixed points, Fix(0). Firstly, the graph of is a La- 
grangian submanifold of M x M, {—00) x u) and its fixed points correspond to the 
intersection points of graph(0) with the diagonal A = {(x, x) ^ M x M}. Thus we 
have the Floer homology of the Lagrangian intersection HF^:{M x M, A, graph{(p)). 
This intersection is transversal if the fixed points of are nondegenerate, i.e. if 1 
is not an eigenvalue of d(j){x), for x G Fix(0). The second approach was mentioned 
by Floer in [17] and presented with details by Dostoglou and Salamon in [6] .We 
follow here Seidel's approach [H] which, comparable with [B], uses a larger class 
of perturbations, but such that the perturbed action form is still cohomologous to 
the unperturbed. As a consequence, the usual invariance of Floer homology under 
Hamiltonian isotopies is extended to the stronger property stated above. Let now 
G Symp'"(M, cj), i.e is monotone. Firstly, we give the definition of HF^{(p) in the 
special case where all the fixed points of are non-degenerate, i.e. for all y G Fix(0), 
det(id— d0j,) 7^ 0, and then following Seidels approach [H] we consider general case 
when has degenerate fixed points. Let Vt^ = {y ^ C°°(]R, M) \ y{t) = tp{y{t + 1))} 
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be the twisted free loop space, which is also the space of sections of T^ — > 5^. The 
action form is the closed one-form a^ on Vt^ defined by 

a^{y)Y = I uj{dy/dt,Y{t))dt. 
Jo 

where y eQ^ andY e TyVL^, i.e. Y{t) E Ty(^t)M and Y{t) = d(j)y(^t+i)Y {t + 1) for all 

The tangent bundle of any symplectic manifold admits an almost complex struc- 
ture J : TM — > TM which is compatible with u in sense that {v,w) = uj{v, Jw) 
defines a Riemannian metric. Let J = {Jt)teR be a smooth path of u;-compatible 
almost complex structures on M such that Jt+i = (t)*Jt- If Y,Y' G TyQ^, then 
Jp uj{Y'{t), JtY(t))dt defines a metric on the loop space Q^. So the critical points 
of a^ are the constant paths in Q^ and hence the fixed points of 0. The negative 
gradient lines of a^j with respect to the metric above are solutions of the partial 
differential equations with boundary conditions 



(2.2) 



u{s,t) = (j){u{s,t + 1)), 
dsU + Jt{u)dtu = 0, 
\ims^±oc u{s,t) e Fix(0) 



These are exactly Gromov's pseudoholomorphic curves 

For y^ G Fix(0), let Ai{y~ ,y^; J, 0) denote the space of smooth maps u : M."^ —^ M 
which satisfy the equations (12.21) . Now to every u G M.{y~ ,y^] J,(f)) we associate 
a Fredholm operator D^ which linearizes (12. 2p in suitable Sobolev spaces. The in- 
dex of this operator is given by the so called Maslov index yu(u), which satisfies 
fi{u) = deg{y~^) — deg{y~) mod 2, where (— l)'^°sy = sign(det(id— d0j,)). We have 
no bubbling, since for surface 7r2(M) = 0. For a generic J, every u G A4{y^ ,y~^; J, 0) 
is regular, meaning that D„ is onto. Hence, by the implicit function theorem, 
•M.k{y~ , y~^ ] J, 4>) is a smooth /c-dimensional manifold, where M.k{y~ ,y^] J,<P) de- 
notes the subset of those u G M.{y~ ,y~^; J, 0) with fi{u) = k E Z. Translation of the 
s- variable defines a free M-action on 1-dimensional manifold j\4i{y~ ,y~^; J,(f)) and 
hence the quotient is a discrete set of points. The energy of a map u : M^ ^ M 
is given by E{u) = f^f^ uj(^dtu{s,t), Jtdtu{s,t)) dtds for all y G Fix(0). P.Seidel 
has proved in [H] that if is monotone, then the energy is constant on each 
A^fc(7/~, ?/"•"; J, 0). Since all fixed points of are nondegenerate the set Fix(0) is 
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a finite set and the Z2-vector space CF^{(f)) := Zg admits a Z2-grading with 

(— 1)'^°^^ = sign(det(id— d0j^)), for all y G Fix(0). The boundedness of the energy 
E{u) for monotone (f) implies that the 0-dimensional quotients A^i(y_, y.)., J, </))/]R 
are actually finite sets. Denoting by ?7,(i/_,?/+) the number of points mod 2 in 
each of them, one defines a differential dj : CF^,{(j)) —^ CF^,_|_i(0) by djy^ = 
"^ n{y^,y^)y+. Due to gluing theorem this Floer boundary operator satisfies 
dj o dj = 0. For gluing theorem to hold one needs again the boundedness of the 
energy E{u) . It follows that {C F^:{(f)) , dj) is a chain complex and its homology is 
by definition the Floer homology of (p denoted HF^f^cf)). It is independent of J and 
is an invariant of (j). 

If has degenerate fixed points one needs to perturb equations fl2.2p in order to 
define the Floer homology. Equivalently, one could say that the action form needs 
to be perturbed. The necessary analysis is given in [51], it is essentially the same 
as in the slightly different situations considered in |B]. But Seidel's approach also 
differs from the usual one in [6] . He uses a larger class of perturbations, but such 
that the perturbed action form is still cohomologous to the unperturbed. 

2.2. Nielsen numbers and Floer homology. 

2.2.1. Periodic diffeomorphisms. 

Lemma 2.1. [22] Let a non-trivial orientation preserving periodic diffeomorphism 
of a compact connected surface M of Euler characteristic x{^) ^ 0. Then each fixed 
point class of (p consists of a single point which has index 1. 

There are two criteria for monotonicity which we use later on. Let uj be an area 
form on M and G Symp(M, uj). 

Lemma 2.2. [20] Assume that every class a G ker(id— 0^,) C Hi(M;7j) is repre- 
sented by a map 7 : S* — *> Fix(0), where S is a compact oriented 1-manifold. Then 
(j) is monotone. 

Lemma 2.3. [20] // (p^ is monotone for some k > 0, then is monotone. If is 
monotone, then (j)^ is monotone for all k > 0. 

We shall say that : M — »■ M is a periodic map of period m, if 0™ is the identity 
map idA/ : M -^ M. 
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Theorem 2.4. [14j If (p is a non-trivial orientation preserving periodic diffeomor- 
phism of a compact connected surface M of Euler characteristic x{^) ^ 0, then (p 
is monotone symplectomorphism with respect to some (p-invariant area form and 

HF,{(P) = Zf '^) 

where N{(f)) denotes the Nielsen number of (p. 

Proof. Let be a periodic diffeomorphisin of least period /. First note that if u) 
is an area form on M, then area form uj := ^j=i(0*)*'^ is 0-invariant, i.e. (p G 
Symp(M, a;). By periodicity of 0, 0' is the identity map idj\,/ : M -^ M. Then from 
Lemmas 12.21 and 12.31 it follows that u can be chosen such that (p G Symp"^(M, a;). 

Lemma 12.11 implies that every y G Fix(0) forms a different fixed point class of 
0, so #Fix(0) = N{(p). This has an immediate consequence for the Floer complex 
{CF^:{(p),dj) with respect to a generic J = {Jt)teR- If Z/^ G Fix(0) are in different 
fixed point classes, then Ai{y~, y~^] J, (p) = 0. This follows from the first equation in 
(12.21) . Then the boundary map in the Floer complex is zero dj = and Z2-vector 
space CF,{(P) := Z*^'"^'^^ = Z^^^\ This immediately implies HF,{(P) = Z^^^'' and 
dim HF^{(p) = N{(p). 

D 

2.2.2. Algebraically finite mapping classes. A mapping class of M is called alge- 
braically finite if it does not have any pseudo-Anosov components in the sense of 
Thurston's theory of surface diffeomorphism.The term algebraically finite goes back 
to J. Nielsen [SH]. 

In [20] the diffeomorphisms of finite type were defined . These are special represen- 
tatives of algebraically finite mapping classes adopted to the symplectic geometry. 

Definition 2.1. [2U] We call (p G Diff+(M) of finite type if the following holds. 
There is a 0- invariant finite union A^ C M of disjoint non-contractible annuli such 
that: 

(1) (p\M \N is periodic, i.e. there exists i > such that 0^|M \ A^ = id. 

(2) Let A^' be a connected component of A^ and £' > be the smallest integer such 
that 0^ maps A^' to itself. Then 0^ |A^' is given by one of the following two models 
with respect to some coordinates {q,p) & I x S^: 
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(twist map) (g,p) i — > {q,p- /(g)) 

(flip-twist map) (g,p) i > {l-q,-p- f{q)), 

where / : / — i> M is smooth and strictly monotone. A twist map is called positive or 
negative, if / is increasing or decreasing. 

(3) Let A^' and i' be as in (2). li i' = 1 and (f)\N' is a twist map, then image(/) C 
[0, 1], i.e. 0|int(A^') has no fixed points. 

(4) If two connected components of A^ are homotopic, then the corresponding local 
models of are either both positive or both negative twists. 



The term fiip-twist map is taken from |31j . 

By Mid we denote the union of the components of M \ int(A^), where restricts 
to the identity. 

The next lemma describes the set of fixed point classes of (p. It is a special case 
of a theorem by B. Jiang and J. Guo [31], which gives for any mapping class a 
representative that realizes its Nielsen number. 

Lemma 2.5 (Fixed point classes). [31] Each fixed point class of (p is either a con- 
nected component of Mid or consists of a single fixed point. A fixed point x of the 
second type satisfies det(id— d0a;) > 0. 

The monotonicity of diffeomorphisms of finite type was investigated in details in 
|20j . Let be a diffeomorphism of finite type and I be as in (1). Then (p^ is the 
product of (multiple) Dehn twists along A^. Moreover, two parallel Dehn twists have 
the same sign, by (4). We say that has uniform twists, if 0^ is the product of only 
positive, or only negative Dehn twists. 

Furthermore, we denote by I the smallest positive integer such that 0^ restricts to 
the identity on M \ A^. 

If uj' is an area form on M which is the standard form dg A dp with respect to the 
(g, p)-coordinates on A^, then u := X]j=i(</'*)*^' i^ standard on A^ and 0-invariant, 
i.e. G Symp(M, tu). To prove that oj can be chosen such that G Symp™(M, cj), 
Gautschi distinguishes two cases: uniform and non-uniform twists. In the first case 
he proves the following stronger statement. 
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Lemma 2.6. [20] // has uniform twists and ui is a (p-invariant area form, then 
0GSymp™(M,a;). 

In the non-uniform case, nionotonicity does not hold for arbitrary 0-invariant area 
forms. 

Lemma 2.7. [20j //</> does not have uniform twists, there exists a (p-invariant area 
form uj such that (j) G Symp'^(M, cj). Moreover, uj can he chosen such that it is the 
standard form dg A dp on N. 

Theorem 2.8. [14j If (f) is a diffeomorphism of finite type of a compact connected 
surface M of Euler characteristic x(^) < and if (p has only isolated fixed points , 
then (j) is monotone with respect to some (p-invariant area form and 

where N{(f)) denotes the Nielsen number of (p. 

Proof. From Lemmas 12.61 and 12.71 it follows that uj can be chosen such that cp G 
Symp"*(M, tu). Lemma 12.51 implies that every y G Fix(0) forms a different fixed 
point class of </>, so 7^Fix(</)) = N{(p). This has an immediate consequence for the 
Floer complex {CF^:{(p),dj) with respect to a generic J = {Jt)teR- If y"^ £ Fix{(p) 
are in different fixed point classes, then Ai{y~,y~^; J,(p) = 0. This follows from 
the first equation in (12.21) . Then the boundary map in the Floer complex is zero 
dj = and Z2-vector space CF^{(p) := Z2 = ^2 • This immediately implies 

n 



Theorem 2.9. [2U] Let (p he a diffeomorphism of finite type, then (p is monotone 
with respect to some (p-invariant area form and 

HFM = ^*(Md, 9m„; Z2) © Zi^^^'''^''^'\ 

Here, L denotes the Lefschetz number. 

Proof. The main idea of the proof is a separation mechanism for Floer connecting 
orbits. Together with the topological separation of fixed points discussed in theorem 
it allows us to compute the Floer homology of diffeomorphisms of finite type. 
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There exists a function H : M —* M. such that H\mt{Mid) is a Morse function, mean- 
ing that all the critical points are non-degenerate and H\{M\Mid) = 0. Let {■ipt)teR 
denote the Hamiltonian flow generated by H with respect to the fixed area form u 
and set <l> := o -i/;!. Then Fix$ = (crit(iJ) fl Mid) U (Fix0 \ M^d). In particular, 
$ only has non-degenerate fixed points. Let A'^o C Mid be a collar neighborhood 
of dMid- Let x~,x~^ G Fix$ fl Mid be in the same connected component of Mid- 
If n G Ai{x~ ,x^] J, $), then imageu C Ms, where Ms denotes the ^-neighborhood 
of Mid \ No with respect to any of the metrics u{., Jt.) [HI [20] Moreover, lemma 
12.51 implies that every y G Fix(0) \ Mid forms a different fixed point class of $. 
This has an immediate consequence for the Floer complex (C-F*(<l>), 9j) with re- 
spect to a generic J = (Jt)fgR. Namely, (CF*($),9j) splits into the subcomplexes 
(Ci, di) and (C2, ^2), where Ci is generated by crit(if) fl Mid and C2 by Fix(0) \ Mid- 
Moreover, C2 is graded by and ^2 = [20]. The homology of (Ci, di) is isomorphic 
to H,{M,d,d+M,d;^2) mm- So HFM ^ //.(M,^, <9+M,,; Z2) © Z*^'^^!^^'''". 
Since every fixed point oi (j)\M \ Mid has fixed point index 1, the Lefschetz fixed 
point formula implies that #(Fix0 \ Mid) = L{<P\{M \ Mid)). 

D 

Remark 2.2. In the theorem 12.81 the set Mid is empty and every fixed point of 
has fixed point index 1 [31]. The Lefschetz fixed point formula implies that 
#Fix0 = N{(j)) = L{(f)) . So, theorem 12.81 follows also from theorem 12.91 

2.2.3. Anosov diffeomorphisms of 2- dimensional torus. The connection between Nielsen 
number and the dimension of symplectic Floer homology remains true for genus 1 
surface and Anosov diffeomorphism . 

Theorem 2.10. [2] If (p is a Anosov diffeomorphism of a 2-dimensional torus T^, 
then (j) is symplectic and 

HF,{4>) = Z?^'^\ 

where N{(f)) = \ det(-E' — 0*)| denotes the Nielsen number of (j) and 0* is an induced 
homomorphism on the fundamental group ofT"^. 

Proof. Hyperbolicity of means that the covering linear map (j) : R^ ^ R^, det = 1 
has no eigenvalue of modulus one. The Anosov diffeomorphism of a 2-dimensional 
torus T^ is area preserving so symplectic. In fact, the covering map has a unique 
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fixed point, which is the origin; hence, by the covering homotopy theorem , the 
fixed points of cf) are pairwise Nielsen nonequivalent.The index of each Nielsen fixed 
point class, consisting of one fixed point, coincides with its Lefschetz index, and 
by the hyperbolicity of 0, the later is not equal to zero. Thus the Nielsen number 
N{(f)) = ^Fix(0). If y^ G Fix(0) are in different Nielsen fixed point classes, then 
Ai{y^,y^] J,(f)) = 0. This follows from the first equation in (12.21) . Monotonicity 
condition trivially follows from the isomorphism H2{M,'R) = H2{T(i,,M.). Then the 
boundary map in the Floer complex is zero dj = and Z2-vector space CF^{(j)) := 

This immediately implies HF^{(f)) = Zg . 

D 



3. Symplectic zeta functions and asymptotic invariant 

3.1. Symplectic zeta functions. Let ModM = '^o{Diff~^{M)) be the mapping 
class group of a closed connected oriented surface M of genus > 2. Pick an ev- 
erywhere positive two-form u on M. A isotopy theorem of Moser [37] says that 
each mapping class of (7 G F, i.e. an isotopy class of Diff^{M), admits represen- 
tatives which preserve u. Due to SeideljH] we can pick a monotone representative 
G Symp'"(M, ci;) of g. Then HF^:{(j)) is an invariant of g, which is denoted by 
HF^{g). Note that HF^{g) is independent of the choice of an area form u by Moser's 
theorem and naturality of Floer homology. By lemma 12.31 symplectomorphisms </>" 
are also monotone for all n > 0. Taking a dynamical point of view, we consider 
the iterates of monotone symplectomorphism (p and define the first symplectic zeta 
function of [H] as the following power series: 



, ^ xjHF^ir)) r. 
X^{z) = exp 2^ z 



n 

, n=l 



where x(-^-^*(0")) is the Euler characteristic of Floer homology complex of 0". Then 
X(i){z) is an invariant of (7, which we denote by Xgi^)- Let us consider the Lefschetz 
zeta function 



L^{z) := exp ^ 



n 

n=l 
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where L(0") := ELo(-l)*'^r fCfc : Hk{M; Q) -^ Hk{M; Q)j is the Lefschetz num- 
ber of (j)^. 

Theorem 3.1. [M] Symplectic zeta function x</>(^) is a rational function of z and 

2 

X<i>{z) = L^(z) = Y[ det (/ - </)*fc.z) * 



^fc+l 



(_1)M 

fc=0 
Proof. If for every n all the fixed points of 0" are non-degenerate, i.e. for all 
X e Fix(0"), det(id-d0"(x)) ^ 0, then we have( see section [2]): x{H F^{(j)'^)) = 
^2,=0n(2,) sign(det(id— d0"(x))) = L^cj)"^). If we have degenerate fixed points one 
needs to perturb equations (12.21) in order to define the Floer homology. The nec- 
essary analysis is given in [JT] is essentially the same as in the slightly different 
situations considered in [B], where the above connection between the Euler charac- 
teristic and the Lefschetz number was firstly established. D 

In [13] we have defined the second symplectic zeta function for monotone sym- 
plectomorphism as the following power series: 

\n=l 

Then F^{z) is an invariant of mapping class g, which we denote by Fg{z). 

Motivation for this definition was the theorem 12.41 and nice analytical properties 
of the Nielsen zeta function 



N^iz) =expij2 



mri^n 



n 

n=l 

, see dni [m iOl [12]. We denote the numbers dimi7F,((/.") by iV„. Let /i(d), d G N, 
be the Mobius function. 

Theorem 3.2. yL4j Let cf) he a non-trivial orientation preserving periodic diffeomor- 
phism of least period m of a compact connected surface M of Euler characteristic 
x{M) < . Then the zeta function F^{z) is a radical of a rational function and 

F^{z) = n (/(T^^^pw, 

d\m 

where the product is taken over all divisors d of the period m, and P{d) is the integer 
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Remark 3.1. Given a symplectomorphism of surface M, one can form the sym- 
plectic mapping torus M^ = T| x 5^, where T| is usual mapping torus . lonel and 
Parker [23] have computed the degree zero Gromov invariants [5B] (these are built 
from the invariants of Ruan and Tian) of Mf and of fiber sums of the Mf with other 
symplectic manifolds. This is done by expressing the Gromov invariants in terms 
of the Lefschetz zeta function L^{z) [25]. The result is a large set of interesting 
non-Kahler symplectic manifolds with computational ways of distinguishing them. 
In dimension four this gives a symplectic construction of the exotic elliptic surfaces 
of Fintushel and Stern [16]. This construction arises from knots. Associated to 
each fibered knot K in S^ is a Riemann surface M and a monodromy diffeomor- 
phism Jk of M. Taking (f) = fx gives symplectic 4-manifolds M^^{K) with Gromov 
invariant Gr{M^^{K)) = yli^(t)/(l - t)^ = L^{t), where Axit) is the Alexander 
polynomial of knot K. Next, let E^{n) be the simply-connected minimal elliptic 
surface with fiber F and canonicla divisor k = {n — 2)F. Forming the fiber sum 
E'^{n,K) = E'^{n)^(^p=j'^)M^^{K) we obtain a symplectic manifold homeomorphic 
to E^{n). Then for n > 2 the Gromov and Seiberg-Witten invariants of E'^{K) are 
Gr{E^{n,K)) = SW{E'^{n,K)) = Ai^(t)(l -t)"-^ [TS1[2S]. Thus fibered knots with 
distinct Alexander polynomials give rise to symplectic manifolds E'^{n, K) which 
are homeomorphic but not diffeomorphic. In particular, there are infinitely many 
distinct symplectic 4-manifolds homeomorphic to E^{n) [16] . 

In higher dimensions it gives many examples of manifolds which are diffeomorphic 
but not equivalent as symplectic manifolds. Theorem 13.11 implies that the Gromov 
invariants of Mf are related to symplectic Floer homology of (p via zeta function 
X(j){z) = L^{z). We hope that the second symplectic zeta function F^{z) give rise to 
a new invariant of symplectic 4-manifolds [16] . 



3.2. Topological entropy and the Nielsen numbers. A basic relation between 
Nielsen numbers and topological entropy h{f) j32j was found by N. Ivanov [27]. We 
present here a very short proof of Jiang of the Ivanov' s inequality. 



Lemma 3.3. 

h{f) >limsup--logA^(r) 

n n 

Proof. Let 5 be such that every loop in X of diameter < 25 is contractible. Let 
e > be a smaller number such that d{f{x), f{y)) < S whenever d{x,y) < 2e. Let 
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En C X he a. set consisting of one point from each essential fixed point class of 
f^. Thus \ En \= Nlj"^). By the definition of h{f), it suffices to show that En is 
(n, e)-separated. Suppose it is not so. Then there would be two points x ^ y E En 
such that d{f^{x), f^{y)) < e ioi o < i < n hence for all i > 0. Pick a path q from 
/*(x) to f\y) of diameter < 2e for o < i < n and let Cn = cq. By the choice of 6 
and e , / o Cj ~ q+i for all i, so /" o cq ^ c„ = Cq. such that This means x, y in the 
same fixed point class of /", contradicting the construction of En- 

n 

This inequality is remarkable in that it does not require smoothness of the map 
and provides a common lower bound for the topological entropy of all maps in a 
homotopy class. 

We recall Thurston classification theorem for homeomorphisms of surfase M of 
genus > 2. 

Theorem 3.4. [46] Every homeomorphism cj) : M ^ M is isotopic to a homeomor- 
phism f such that either 

(1) f is a periodic map; or 

(2) f is a pseudo-Anosov map, i.e. there is a number A > 1 (stretching factor) and 
a pair of transverse measured foliations {E'^,n'^) and (F",/i") such that f{F^,n'^) = 
{F', i/i") and f{F'',^i'') = (F", A/i"); or 

(3)f is reducible map, i.e. there is a system of disjoint simple closed curves 7 = 

{71, ,7fc} in intM such that 7 is invariant by f (but '-fi may be permuted) and 7 

has a f -invariant tubular neighborhood U such that each component of M \U has 
negative Euler characteristic and on each(not necessarily connected) f -component 
ofM\ U, f satisfies (1) or (2). 

The map / above is called a Thurston canonical form of (j). In (3) it can be chosen 
so that some iterate /"^ is a generalised Dehn twist on U . Such a / , as well as the 
/ in (1) or (2), will be called standard. A key observation is that if / is standard, 
so are all iterates of /. 

Lemma 3.5. [9] Let f be a pseudo-Anosov homeomorphism with stretching factor 
X > 1 of surfase M of genus > 2. Then 

M/) = /o(?(A)=limsup-- log iV(r) 
n n 
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Lemma 3.6. ^U\ Suppose f is a standard homeomorphism of surfase M of genus 
> 2 and A is the largest stretching factor of the pseudo-Anosov pieces( X = 1 if there 
is no pseudo-Anosov piece). Then 

/i(/) = /o^(A) = lim sup -■ log iV(r) 
n n 

3.3. Asymptotic invariant. The growth rate of a sequence a„ of complex numbers 

is defined by 

Growth(a„) := ?72ax{l, limsup Iflnr"} 

n— ►oo 

which could be infinity. Note that Growth(a„) > 1 even if all a„ = 0. When 
Growth(a„) > 1, we say that the sequence a„ grows exponentially. 

Definition 3.2. We define the asymptotic invariant F°°{g) of mapping class g G 
Modu = T^o{.Dif f^{M)) to be the growth rate of the sequence {a^ = dimifF*(0'^)} 
for a monotone representative G Symp'"(M, cj) of g: 

F^{g) := Growth(dim/7F,(0")) 

Example 3.3. If is a non-trivial orientation preserving periodic diffeomorphism 
of a compact connected surface M of Euler characteristic xiM) < , then the 
periodicity of the sequence dira H F^:{(f)^) implies that for the corresponding mapping 
class g the asymptotic invariant 

F^{g) ■= Growth(dimi7F,(0")) = 1 

Example 3.4. Let be a monotone diffeomorphism of finite type of a compact 
connected surface M of Euler characteristic x{^) < ^'^^ 9 ^ corresponding al- 
gebraically finite mapping class. Let U be the open regular neighborhood of the 

k reducing curves 71, ,7/; in the Thurston theorem, and Mj be the component 

of M \ f/.Let F be a fixed point class of </>. Observe from [HI] that if F C Mj, 
then ind(F, (p) = ind(F, (j)j).So if F is counted in N{(()) but not counted in ^ • N{(j)j) 
, it must intersect U. But we see from [21] that a component of U can intersect 
at most 2 essential fixed point classes of 0. Hence we have N{(f)) < "^j N{(f)j). For 
the monotone diffeomorphism of finite type maps 0j are periodic. Applying last 
inequality to 0" and using remark [23] we have 

< dimHF^ir) = dimi7,(Mif \ 5Mjf ); Z2) + A(0"|(M \ Mjf ))) < 



FLOER HOMOLOGY AND THE POINCARE - BIRKHOFF THEOREM 17 

< dim H.{Mlf, dM^2^-Z2) + iV(</.") < 

dimi7,(MiJ\ SMjf ); Z2) + ^ iV((0,)") + 2k< Const 

j 
by periodicity of (pj. Taking the growth rate in n, we get that asymptotic invariant 
F^{g) = 1. 

Remark 3.5. For pseudo-Anosov mapping class g G Modu = '^o{Diff^{M)) we 
have 

dim HF,{g) > N{g), F°^{g) > hmsup |Ar(^")|^/" = h{tlj) = A > 1, 

n— >oo 

where N{g) denotes the Nielsen number oi g and ^/^ is a standard(Thurston canonical 
form) representative of g. 

Remark 3.6. For any mapping class g G Modu = '^o{Diff~^{M)) we have 

dimHFM>N{g), F^{g)>X, 

where A is the largest stretching factor of pseudo-Anosov pieses of a standard( 
Thurston canonical form) representative oi g { X := 1 ii there is no pseudo-Anosov 
piece). 

4. a generalisation of the poincare - birkhoff theorem and 
Arnold conjecture. Concluding remarks 

A natural generalization of the Poincare- Birkhoff theorem concerns the estimation 
of the number of fixed points of symplectomorphism G Symp(M^"',Ci;). Actually 
this question was already raised by Birkhoff in pp. He wrote with reference to 
the Poincare - Birkhoff theorem: "Up to present time no proper generalisation of 
this theorem to higher dimensions has been found, so that its applications remains 
limited to dynamical systems with two degrees of freedom [1], page 150." 

Based on the Poincare - Birkhoff theorem Arnold formulated in the 1960s his fa- 
mous conjecture: a Hamiltonian symplectomorphism ( time-1 map of a time- dependent 
Hamiltonian flow) should have at least as many fixed points as a function on the 
manifold have critical points. 

Let (j) : M^" —>■ M ^" be a Hamiltonian symplectomorphism of a compact symplec- 
tic manifold {M'^^,uj). In the case when all fixed points of are nondegenerate the 
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Arnold conjecture asserts that 

where 2n = dimM, bk{M) = dim if ^(M, Q). 

The Arnold conjecture in the nondegenerate case has now been proved in full 
generality. It was first proved by Eliashberg [S] for Riemann surfaces. For tori of 
arbitrary dimension it was proved in the celebrated paper by Conley and Zehnder[2]. 
The most important breakthrough was Floer's proof of the Arnold conjecture in the 
nondegenerate case for monotone symplectic manifolds [18]. His proof was based 
on Floer homology. His method has been pushed through by Fukaya-Ono|19j . Liu- 
Tian[31] and Hof er- S alamon [22] to establish the nondegenerate case of the Arnold 
conjecture for all symplectic manifolds. 

Some progress has been made with the original Arnold conjecture by Rudyak [13], 
using a development of Lusternik - Schnirelman theory called category weight. Using 
these ideas it was proved in [H] that the original conjecture holds in the case where 
degeneracies are allowed, provided that both classes [u] and ci vanish on 7r2(M^"). 

The Hamiltonian symplectomorphism is isotopic to identity map idM- In this 
case all fixed points are in the same Nielsen fixed point class. The Nielsen number 
of is or 1 depending on Lefschetz number is or not. So, the Nielsen number 
is very weak invariant to estimate the number of fixed points of for Hamiltonian 
symplectomorphism. From another side, as we saw in theorem 12.41 for the nontrivial 
periodic symplectomorphism of a surface, the Nielsen number of gives an exact 
estimation from below for the number of nondegenerate fixed points of 0. These 
considerations lead us to the following question 

Question 4.1. How to estimate the number of fixed points of symplectomorphism 

(not necessary Hamiltonian) G Symp(M^"',ci;) which has only nondegenerate fixed 
points? 

This question can be considered as a generalisation of the Poincare - Birkhoff 
theorem and as a generalisation of the Arnold conjecture in nondegenerate case . 

4.1. Algebraically finite mapping class. If ^ is a diffeomorphism of finite type 
of surface M then ip G Symp"*(M, u) for some -^-invariant form u. Suppose now that 
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symplectomorphism has only non-degenerate fixed points and is Hamiltonian 
isotopic to ip. Then </> G Symp™'(M, u;) and HF^:{(f)) is isomorphic to HF^:{ip). So, 
from theorem 12.91 it follows that 

i^Fix{(f)) > dim HF^{(j)) = dim HF^{ip) = 

= dimi7,(M^=id, 9M^=id; Z2) + N{i,\{M \ M^=id)) = 

2 
= Y, h{M^=i<i, 5M^=id; Z2) + N{^\{M \ M^=id)) 

fc=0 

This estimation can be considered as a generahsation of the Poincare - Birkhoff 
theorem and as a generahsation of Arnold conjecture for symplectomorphism with 
nondegenerate fixed points in algebraically finite mapping class, becouse it implies 
Arnold conjecture for ip = id. If ip is nontrivial orientation preserving periodic 
diffeomorphism then by theorem 12.41 ip is a. monotone symplectomorphism and by 
lemma 12.11 it has only nondegenerate fixed points. The theorem 12.41 implies an 
estimation 

#F2x(0) > dim HF^{(j)) = dim HF^iip) = N{'4;) 

This estimation can be considered as a generalisation of Poincare - Birkhoff theorem 
and Arnold conjecture for a nontrivial orientation-preserving periodic mapping class. 

4.2. Pseudo-Anosov mapping class. For pseudo-Anosov "diffeomorphism" ijj in 
given pseudo-Anosov mapping class g we also have, as in theorems 12.4112.81 and 12. 9[ a 
topological separation of fixed points [IHl [3T|, [26], i.e the Nielsen number of pseudo- 
Anosov "diffeomorphism" ip equals to the number of fixed points of ip and there are 
no connecting orbits between them. But we have the following difficulties. Firstly, a 
pseudo-Anosov "diffeomorphism" is a smooth and a symplectic automorphism only 
on the complement of the singular set of a pair of transverse measured foliations. 
Secondly, in the case of a pseudo-Anosov "diffeomorphism" we have to deal with fixed 
singular points of index 1 —p where p > 3 is a number of prongs of fixed singularity 
. Such fixed points are degenerate from symplectic point of view and therefore need 
a local perturbation. The following theorem can be considered as a generalisation 
of Poincare - Birkhoff theorem for a symplectomorphism with nondegenerate fixed 
points in a given pseudo-Anosov mapping class. 
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Theorem 4.1. (Cotton- Clay [3], Fel'shtyn) If (p is a symplectomorphism with non- 
degenerate fixed points in given pseudo-Anosov mapping class {</>} = g = {^}, then 

HF,{<P) = HF,{g) = zf ^^^'^(^' l'"''^"^! 

and 

#Fix{(f)) > dim HF, (0) = dim HF, (g) = J^ I I^d(^) I ' 

x'eFix(-!/') 

where ip is a canonical pseudo-Anosov representative of g. 

Proof. We describe main steps of the proof. Firstly we smooth the singular pseudo- 
Anosov map tp locally near the singularities using Hamiltonian vector fields to get 
symplectomorphism tp in the given mapping class g with |Ind(a;)| nondegenerate 
fixed points in the small neighborhood of x. The idea of such smoothing was sug- 
gested to the author by Kaoru Ono in 2005. Full details of this smoothing can 
be found in [3]. Then we show that all | Ind(a;)| nondegenerate fixed points in the 
small neighborhood of x are in the same Nielsen fixed point class. On the next 
step we prove that there are no pseudoholomorphic curves of index 1 between these 
nondegenerate fixed points. Then the boundary map in the Floer complex of ip is 
zero and Z2-vector space CF^{ip) := Z* = 1^2 "'^^"'''''' . This immediately 

implies that HF^iip) is well defined and HF^{g) = HF^{ip) ^ ^f -eFix(^) |ind(x)| ^ ^^^ 
dim. HF^{ip) = JZx&FixM) I Iiid(x)|. On the last step of the proof, which was fully jus- 
tified only recently by A. Cotton-Clay [3] , we show that for any symplectomorphism 
in a pseudo-Anosov mapping class g with nondegenerate fixed points HF^{(p) is 
well defined and HF^{(p) = HF^{g). 

n 

Remark 4.2. A. Cotton-Clay found in [5] a nice combinatorial formula computing 

HF^.{%p) using train tracs . 

Remark 4.3. In [7j Floer homology were calculated for certain class of pseudo- 
Anosov maps which are compositions of positive and negative Dehn twists along 
loops in M forming a tree-pattern. It is interesting to compare the results above 
with this calculation. 
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4.3. Reducible mapping class. Recently, in the paper [3], A. Cotton-Clay calcu- 
lated Seidel's symplectic Floer homology for reducible mapping classes. This result 
completing previous computations in the case of arbitrary compositions of Dehn 
twists along a disjoint collection of curves [H], in the case of periodic mapping 
classes [20| [T3], as well as reducible mapping classes in which the map on each 
component is periodic [20] and in the case of certain compositions of Dehn twists, 
including some pseudo-Anosov maps [7j. 

In the case of reducible mapping classes a energy estimate forbids holomorphic 
discs from crossing reducing curves except when a pseudo-Anosov component meets 
an identity component ( with no twisting). Let us introduce some notation follow- 
ing [3]. Recall the notation of Mid for the collection of fixed components as well 
as the tree types of boundary: 1) d+Mi^, d^Mi^ denote the collection of compo- 
nents of dMid on which we've joined up with a positive (resp. negative) twist; 2) the 
collection of components of dMi^ which meet a pseudo-Anosov component will be 
denoted dpMi^. Additionally let Mi be the collection of periodic components and 
let M2 bethe collection of pseudo-Anosov components with punctures( i.e. before 
any perturbation) instead of boundary components wherever there is a boundary 
component that meets a fixed component. We further subdivide Mid. Let Ma be the 
collection of fixed components which don't meet any pseudo-Anosov components. 
Let Mft p be the collection of fixed components which meet one pseudo-Anosov com- 
ponent at a boundary with p prongs. In this case, we assign the boundary com- 
ponents to (9-|_Mid (this is an arbitrary choice). Let M^ be the collection of the 
Mfy^p with each component punctured once. Let M^^g be the collection of fixed com- 
ponents which meets at least two pseudo-Anosov components such that the total 
number of prongs over all the boundaries is q. In this case, we assign at least one 
boundary component to d^M^^ and at least one to 9„Mid (and beyond that, it does 
not matter). 

Theorem 4.2. (A. Cotton-Clay [3]j Let cf) he a perturbed standard form map [3] in 
a reducible mapping class g with choices of the signs of components of dpM^^.Then 
HF^,{(f)) is well-defined and 

HF^g) = HF,{<P) = H,{Ma, d+M-.^; Z2)© 

®piH,{M^,d+Mh;Z2) © (Z2)(P^i)l'^°(^''-'')l)© 
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where L(0|Mi) is the Lefschetz number of(p\Mi, the Zg summand is all in even 

degree, the other two Z2 summands are all in odd degree, and C F^{(f)\M2) denotes 
the chain complex for (p on the components M2 

Remark 4.4. The first summand and the Z2 are as in R. Gautschi's Theorem 

12.91 |20j . The last summand comes from the fact that there are no differentials in the 
Floer chain complex on the pseudo-Anosov components as in the proof of Theorem 
14. 1[ The sums over p and q arise in the same manner as the first summand. 

Corollary 4.3. As an application, A. Cotton-Clay gave recently [11[5] a sharp lower 
bound on the number of fixed points of an area-preserving map( with nondegenerate 
fixed points) in any prescribed mapping class(rel boundary). 

4.4. Concluding remarks. 

• Due to P. Seidel [12] dimiJF^,(</)) is a new symplectic invariant of a four- 
dimensional symplectic manifold with nonzero first Betti number. This 
4-manifold produced from symplectomorphism by a surgery construc- 
tion which is a variation of earlier constructions due to McMullen-Taubes, 
Fintushel- Stern and J. Smith. We hope that our asymptotic invariant and 
symplectic zeta function F^{z) also give rise to a new invariants of contact 
3- manifolds and symplectic 4-manifolds. 

• Nielsen Floer homology. Let : M^" -^ M^" be a symplectomorphism 
of a compact symplectic manifold (M^",Ci;). Suppose that all fixed points of 
are nondegenerate. The symplectomorphism has finite number of Nielsen 
fixed point classes Fi, F2, ...., -F„ and there is no pseudoholomorphic curves( 
instantons) between fixed points in the different Nielsen fixed point classes 
because there is no Nielsen discs between them. This simple observation 
strongly suggests to consider for every i a local Floer complex CF^{(f),Fi) 
generated by nondegenerate fixed points in the Nielsen fixed point class Fi 
and to define for every i a local Nielsen Floer homology NFH^,{(f),Fi) and 
then to define Nielsen Floer homology NFH^{(j)) of symplectomorphism as 
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direct sum over all Nielsen fixed point classes of local Nielsen Floer homology: 

NFH,{(t)) = ®F,NFH,{(t),Fi). 

Question 4.5. Under which conditions on G Symp{M'^^,uj) Nielsen Floer 
homology are well-defined? 

The results discussed in the present article strongly suggest that a calcu- 
lation of Nielsen Floer homology and investigation of their properties should 
be important for a higher dimensional generalisation of Poincare-Birkhoff 
theorem and Arnold conjecture in nondegenerate case. The author came to 
the idea of Nielsen Floer homology in 2001 after discussions with V. Turaev. 
A lack of a weak version of a monotonicity for the symplectomorphism pre- 
vented the development of this theory at that moment. Recently, Andrew 
Cotton-Clay [3] defined weak monotonicity for a given Nielsen fixed point 
class F of symplectomorphism of surface and proved that Nielsen Floer ho- 
mology are well defined in this case. The properties of F-weak monotone 
symplectomorphism of surface play a crucial role in his computation of Sei- 
del's symplectic Floer homology for reducible mapping classes. 
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